Large language models and bariatric surgery patient education: a comparative readability analysis of GPT-3.5, GPT-4, Bard, and online institutional resources.

BACKGROUND: The readability of online bariatric surgery patient education materials (PEMs) often surpasses the recommended 6th grade level. Large language models (LLMs), like ChatGPT and Bard, have the potential to revolutionize PEM delivery. We aimed to evaluate the readability of PEMs produced by U.S. medical institutions compared to LLMs, as well as the ability of LLMs to simplify their responses. METHODS: Responses to frequently asked questions (FAQs) related to bariatric surgery were gathered from top-ranked health institutions. FAQ responses were also generated from GPT-3.5, GPT-4, and Bard. LLMs were then prompted to improve the readability of their initial responses. The readability of institutional responses, initial LLM responses, and simplified LLM responses were graded using validated readability formulas. Accuracy and comprehensiveness of initial and simplified LLM responses were also compared. RESULTS: Responses to 66 FAQs were included. All institutional and initial LLM responses had poor readability, with average reading levels ranging from 9th grade to college graduate. Simplified responses from LLMs had significantly improved readability, with reading levels ranging from 6th grade to college freshman. When comparing simplified LLM responses, GPT-4 responses demonstrated the highest readability, with reading levels ranging from 6th to 9th grade. Accuracy was similar between initial and simplified responses from all LLMs. Comprehensiveness was similar between initial and simplified responses from GPT-3.5 and GPT-4. However, 34.8% of Bard's simplified responses were graded as less comprehensive compared to initial. CONCLUSION: Our study highlights the efficacy of LLMs in enhancing the readability of bariatric surgery PEMs. GPT-4 outperformed other models, generating simplified PEMs from 6th to 9th grade reading levels. Unlike GPT-3.5 and GPT-4, Bard's simplified responses were graded as less comprehensive. We advocate for future studies examining the potential role of LLMs as dynamic and personalized sources of PEMs for diverse patient populations of all literacy levels.

Perceptions and Expectations Regarding the Step 1 Score Change Among Surgical and Non-Surgical Program Directors: A Systematic Review and Meta-Analysis.

There are currently no studies examining differences in perceptions and expected impact of the Step 1 score change to pass/fail between surgical and non-surgical program directors (PDs). We conducted a systematic review in May 2023 of PubMed, Scopus, Web of Science, and PSYCInfo to evaluate studies examining PDs' perspectives regarding the Step 1 score change. We performed random-effects meta-analyses to determine differences in perspectives among surgical and non-surgical PDs. Surgical PDs (76.8% [95% CI, 72.1%-82.0%], I2 = 52%) reported significantly greater rates of disagreement with the score change compared to non-surgical (65.1% [95% CI, 57.9%-73.1%], I2 = 69.7%) (P = .01). Surgical PDs also reported significantly greater rates of agreement that the score change will increase the difficulty in objectively comparing applicants (88.1% [95% CI, 84.6%-91.7%], I2 = 16.4%), compared to non-surgical (81.0% [95% CI, 75.6%-86.8%], I2 = 72.6%) (P = .04). There was less heterogeneity among non-surgical PDs (88.7% [95% CI, 86.2%-91.2%], I2 = 0%), compared to surgical (84.7% [95% CI, 79.0%-90.8%], I2 = 67.3%), regarding expected increases in emphasis on Step 2, although the difference in rates of agreement was not statistically significant. Overall, there is significant heterogeneity in the literature regarding expected changes in the residency application review process. Most PDs reported significant disagreement with the score change, greater expected difficulty in objectively evaluating applicants, and greater emphasis on Step 2, with surgical PDs reporting greater rates of disagreement, greater expected difficulty, and heterogeneity regarding expected increases in emphasis on Step 2, compared to non-surgical. Additionally, there is significant heterogeneity in the overall literature regarding expected changes in the residency application review process. Further research is needed to establish evidence-based guidelines that improve the overall residency application process for all stakeholders.

Providers' Knowledge and Perceptions of Bariatric Surgery: a Systematic Review.

Bariatric surgery remains underutilized despite its proven efficacy in the management of obesity. Provider perceptions of bariatric surgery are important to consider when discussing utilization rates. PubMed, SCOPUS, and OVID databases were searched in April 2023, and 40 published studies discussing providers' knowledge and perceptions of bariatric surgery were included. There were generally positive perceptions of the efficacy of bariatric surgery, although overestimations of surgical risks and postoperative complications were common. Providers' previous training was associated with knowledge and perceptions of bariatric surgery and familiarity with perioperative management across studies. These perceptions were also associated with referral rates, suggesting that inadequate provider knowledge may contribute to bariatric surgery underutilization. We advocate for increased bariatric surgery-related education throughout all stages of medical training and across specialties.

Assessing the Accuracy of Responses by the Language Model ChatGPT to Questions Regarding Bariatric Surgery.

PURPOSE: ChatGPT is a large language model trained on a large dataset covering a broad range of topics, including the medical literature. We aim to examine its accuracy and reproducibility in answering patient questions regarding bariatric surgery. MATERIALS AND METHODS: Questions were gathered from nationally regarded professional societies and health institutions as well as Facebook support groups. Board-certified bariatric surgeons graded the accuracy and reproducibility of responses. The grading scale included the following: (1) comprehensive, (2) correct but inadequate, (3) some correct and some incorrect, and (4) completely incorrect. Reproducibility was determined by asking the model each question twice and examining difference in grading category between the two responses. RESULTS: In total, 151 questions related to bariatric surgery were included. The model provided "comprehensive" responses to 131/151 (86.8%) of questions. When examined by category, the model provided "comprehensive" responses to 93.8% of questions related to "efficacy, eligibility and procedure options"; 93.3% related to "preoperative preparation"; 85.3% related to "recovery, risks, and complications"; 88.2% related to "lifestyle changes"; and 66.7% related to "other". The model provided reproducible answers to 137 (90.7%) of questions. CONCLUSION: The large language model ChatGPT often provided accurate and reproducible responses to common questions related to bariatric surgery. ChatGPT may serve as a helpful adjunct information resource for patients regarding bariatric surgery in addition to standard of care provided by licensed healthcare professionals. We encourage future studies to examine how to leverage this disruptive technology to improve patient outcomes and quality of life.

Patient and the Public's Perceptions of Bariatric Surgery: A Systematic Review.

INTRODUCTION: Bariatric surgery is an effective therapeutic modality for obesity and related comorbidities, yet it remains significantly underutilized. Patient perceptions and expectations may influence the decisions of eligible patients in pursuing surgery. METHODS: PubMed, SCOPUS, and OVID databases were searched in July 2022 to identify published studies discussing patient and the public's perceptions of bariatric surgery. RESULTS: The literature shows participants often reported bariatric surgery to be a life-changing intervention known to induce weight loss, improve obesity-related comorbidities, and improve quality of life. However, a significant proportion of survey respondents perceived bariatric surgery as unsafe or risky. Patients belonging to racial minority groups cited higher concern with mortality risk, lower weight loss expectations, and different motivations to pursue bariatric surgery. Female patients were significantly more likely to have more positive perceptions of, and higher expectations of weight loss from, bariatric surgery. CONCLUSIONS: The literature highlighted discordance between patient perceptions and the demonstrated clinical safety and efficacy profile of bariatric surgery. Overestimations of the risks, unrealistic expectations, and unfamiliarity with bariatric surgery outcomes were common findings. These perceptions of bariatric surgery may contribute to its underutilization among eligible patients. Perceptions and motivations often varied by race, region, sex, and age, which demonstrates the necessity of patient-centered education in the prereferral stage. The literature also demonstrated misconceptions of bariatric surgery among the public. Further research should explore the impact of education on the perceptions of patients and the public.

Exploratory study of the provision of academic and health-related accommodations to transition-age adolescents and emerging adults with spina bifida.

PURPOSE: The purpose of this exploratory study was to investigate the types of academic and health-related accommodations provided to adolescents and emerging adults with spina bifida aged 9-20 years. METHODS: Data were extracted from the paper and electronic records of transition-age youth enrolled in the study. Four open ended items involved content analysis. RESULTS: The most frequently identified accommodation was enrollment in special education classes in 47.7% of the charts. Other academic accommodations that were most often reported were adaptive physical education (n = 71, 39.9%), tutoring (n = 28; 15.7%), and home schooling (n = 21; 11.8%). Clean intermittent catheterization was the most frequently identified health-related accommodation provided by the school nurse/aide (n = 57; 32%).The largest percentage of requests for additional accommodations were made during the middle school grades (15; 54.8%) followed by high school (10; 32.2%). CONCLUSION: Findings demonstrated that persistent issues were identified by parents/adolescents regarding the provision of school-related accommodations. This is a relevant area for clinical practice to ensure students with special health care needs and those with spina bifida receive the academic and health-related accommodations in their Individualized Education Program/504 plans.